Cognitive engineering graph

ABSTRACT

A method for representing knowledge in a cognitive engineering system (CES) includes receiving information relating to an automation engineering project from an engineering tool, storing the received information in a cognitive engineering graph (CEG) storing a plurality of previously generated CEGs for previous automation engineering projects, and establishing a communication path between the CEG storing the received information and the plurality of previously generated CEGs. The method may further include applying machine learning to the stored CEG based on the received information and the stored plurality of previously generated CEGs. The machine learning may analyze the CEG to identify at least one pattern that is representative of a given object from the automation engineering project. The CES may automatically add an element to the CEG based on the received information and a query from a user. Further, the user may request a change made by the CES be reversed.

TECHNICAL FIELD

This application relates to automation engineering. More particularly,the application relates to cognitive engineering for automation systems.

BACKGROUND

Performing automation engineering tasks requires a high level of humantechnical and domain expertise due to the complexity and critically ofmodern automation systems in manufacturing and assembly, chemical,pharmaceutical, food and beverage, paper, electronics, etc.) The growingcomplexity of automated systems, the increasing requirements for highproductivity and quality of the engineering tasks, and the increasingdemands for safety and high availability of the automation make it verydifficult for teams of human experts to write automation programs fastenough. Rotation of staff aggravates this problem for organizations.Improved systems and methods to address these challenges is desired.

SUMMARY

According to some embodiments of this disclosure, a method forrepresenting knowledge in a cognitive engineering system (CES) includesreceiving information relating to an automation engineering project froman engineering tool, storing the received information in a cognitiveengineering graph (CEG) comprising a plurality of nodes representativeof an element of the automation engineering project and at least on edgeconnecting two of the nodes, the at least one edge representative of arelationship between the connected nodes, storing a plurality ofpreviously generated CEGs representative of other prior automationengineering projects, and establishing a communication path between theCEG storing the received information and the plurality of previouslygenerated CEGs. In some embodiments, the method may further includeapplying machine learning to the stored CEG based on the receivedinformation and the stored plurality of previously generated CEGs. Themachine learning may be used to analyze the CEG based on the receivedinformation to identify at least one pattern that is representative of agiven object of interest from the automation engineering project. Insome embodiments the CES may automatically add an element to the CEGbased on the received information and on a query from a user. Accordingto an embodiment, the user may request that a change made by the CES bereversed. An undo action may be performed where the system identifiesany recent automatic changes and any associated dependencies and removesthose changes returning the system to the state it was in prior theautomatic changes being performed.

The knowledge representation in the form of the CEG may include nodesthat represent physical objects in the automation engineering project oran automation program for controlling a corresponding physical object inthe automation engineering program. The CEG may include a representationof a human machine interface and/or a programmable logic controller. TheCEG may be used to validate a design for the automation engineeringproject by comparing the CEG to a plurality of previously generatedCEGs. I some embodiments the generated CEG can be compared to previouslygenerated CEGs and provide a suggested course of action to a user.

According to some embodiments of the present disclosure, a system forproviding a knowledge representation in a cognitive engineering system(CES) includes a computer-based engineering tool for providing at leastone of designing, programming simulation and testing of an automationsystem, a cognitive system in communication with the computer-basedengineering tool comprising: a knowledge extraction module foridentifying and storing information contained in a project of thecomputer-based engineering tool and from data received from a physicalautomation system, a machine learning module for analyzing knowledgeextracted by the knowledge extraction module and identifyingcharacteristics of the automation system; an inductive programmingmodule of automatically generating control programs for the automationsystem based on the stored information from the knowledge extractionmodule, and a knowledge representation comprising a cognitiveengineering graph (CEG), the CEG comprising a plurality of nodesrepresentative of an element of the automation engineering project andat least on edge connecting two of the nodes, the at least one edgerepresentative of a relationship between the connected nodes.

The system may include a computer memory that stores a plurality of CEGsfrom previously designed projects in communication with the machinelearning module for analyzing past knowledge. Based on analysis of theCEG of a current design in view of the previously designed projects, afeedback module may be provided to give information to the user. Forexample, a user may be provided with feedback relating to the validationof the project being designed. In other embodiments, the feedback modulemay provide a user with a recommended course of action based in part onthe actions taken in previous projects.

A communications channel may be established between the knowledgeextraction module and a physical automation system. The physicalautomation system generates data relating to the operating state of theautomation system and provides the information in the data to theknowledge representation.

An automated reasoning module may be in communication with theengineering tool and the knowledge representation and may be configuredto make certain design decisions including the automatic addition of acomponent to an engineering project in the engineering tool.

BRIEF DESCRIPTION OF THE DRAWINGS

The foregoing and other aspects of the present invention are bestunderstood from the following detailed description when read inconnection with the accompanying drawings. For the purpose ofillustrating the invention, there is shown in the drawings embodimentsthat are presently preferred, it being understood, however, that theinvention is not limited to the specific instrumentalities disclosed.Included in the drawings are the following Figures:

FIG. 1 is a block diagram of a cognitive automation engineering systemaccording to aspects of embodiments of this description.

FIG. 2 is a representative example of a display screen that may be usedas part of an automation engineering tool for designing a cognitiveautomation engineering system according to aspects of embodiments ofthis description.

FIG. 3 is a cognitive engineering graph for an automation systemaccording to aspects of embodiments of this description.

FIG. 4 is a pattern extracted from a cognitive engineering graph for anautomation system according to aspects of embodiments of thisdescription.

FIG. 5 is the pattern of FIG. 4 with added nodes to represent a newelement of an automation system according to aspects of embodiments ofthis description.

FIG. 6 is the simplified pattern of FIG. 5 for display to a useraccording to aspects of embodiments of this description.

FIG. 7 is the simplified view of FIG. 6 with new user-defined propertiesaccording to aspects of embodiments of this description.

FIG. 8 shows an updated pattern of FIG. 5 including new user-definedproperties according to aspects of embodiments of this description.

FIG. 9 shows an updated cognitive engineering graph for a systemincluding new user-defined properties according to aspects ofembodiments of this description.

FIG. 10 is a block diagram of a computer system that may be used toimplement a cognitive engineering system and cognitive engineering graphaccording to aspects of embodiments of the present disclosure.

FIG. 11 is a block diagram of a cognitive engineering graph thatprovides varying level of detail for objects represented by thecognitive engineering graph

FIG. 12 is a block diagram for a method of representing knowledge in acognitive engineering system according to aspects of embodiments of thepresent disclosure.

DETAILED DESCRIPTION

Embodiments described in this disclosure address bringing artificialintelligence (AI) to automation engineering tools to assist humans indesign while increasing their productivity and ensuring high quality.The result is generating automation programs that provide higherreliability, availability and safety. Typically, if the complexity of anautomation engineering task is significant, additional human engineersare added to the design team with the hope of providing solutions to theproblem that are timely and contain high quality. Unfortunately, addingmore humans to the teams does not scale in practice due to the increasedoverhead in communication and management.

FIG. 1 is a high-level diagram of a Cognitive Automation EngineeringSystem (CES) 110 configured to solve the problem that conventionalautomation engineering systems fail to leverage machine intelligence toimprove decisions. In the CES 110, the engineer 101 provides human input103 using an engineering tool 111, denoted as automation engineeringsystem. The engineering tool 111 allows the engineer 101 to providefunctionality including but not limited to design, configuration,programming, simulation and testing. When a design is complete, theengineering tool 111 allows the design to be deployed 125 in a physicalautomation system 120.

Cognitive engineering utilizes machine learning to allow the cognitiveengineering system 110 to assist the engineer 101. Through machinelearning, the CES 110 can recognize aspects of the physical automationsystem 120, along with past engineering actions previously taken and usethis knowledge to assist the engineer 101 during design. For example,the CES 110 may observe an action the engineer 101 performs in theengineering tool 111 and associate the action with an action previouslytaken by the same or another engineer. Based on this recognition, theCES 110 may suggest to the engineer 101 that some or all of the projectcould be preloaded by the CES 110 into the workspace of the engineeringtool 111. In some embodiments, the CES may make suggestions to theEngineer 101 regarding future design actions. For example, the CES 110may observe the engineer 101 adding a system component to theengineering tool. The CES 110 may compare the action to actions taken byother engineers in the past who were working on the same problem or in asimilar system. If the engineer's action is aligned with actions the CES110 has seen as typical in the past, the CES 110 may suggest to theengineer 111 that a different course of action may be considered. Theengineer has the option of accepting the CES suggestion or continuingwith the original action of the engineer 101.

The cognitive system 113 leverages captured knowledge to assist theengineer 101. The cognitive system 113 functions to extract knowledge119 from the system via system input 127 and provide a storedrepresentation of the collected knowledge 117. Machine learning 114 isperformed on the stored extracted knowledge to identify and exploitrelationships in the data. In some embodiments the knowledge may be usedto provide inductive programming 112 for components or control of thesystem. Automated reasoning 115 is applied to the knowledgerepresentation 117 and may be used to analyze the knowledgerepresentation 117 and provide feedback to the engineer 101 based on theanalysis.

With reference to FIG. 1 , embodiments of the present invention aredirected to achieving a realization of the “Knowledge Representation”component 117 of the Cognitive Automation Engineering System 110.Through the reuse of engineering knowledge in the knowledgerepresentation 117, a Cognitive Automation Engineering system 110 canimprove the productivity and the quality of automation engineeringprojects.

The representation of knowledge has long been a topic of research withearly attempts focusing on maintaining knowledge in a standard orspecialized database, such as a relational database using databaselanguages such as SQL. These databases store information in the form ofrules and facts. In later research, the use of knowledge graphs hasbecome popular for representing and analyzing linked information fromcommunication networks, social network logistics, marketing systems andgeographical information.

This disclosure presents a Cognitive Engineering Graph (CEG) as the mainbuilding block of a Cognitive Engineering System (CES), responsible forthe knowledge representation 117 of the system. The CEG represents allrelevant information about the engineering process in the form of agraph, in accordance with the techniques used by today's graph databases(e.g., Neo4j, BlazeGraph).

The CEG is created and subsequently updated with information obtainedfrom engineering tools such as TIA Portal. Other related engineeringtools such as computer assisted design (CAD)/computer aided engineering(CAE)/computer assisted manufacturing (CAM) may also provide informationto the CEG. The information is analyzed and represented in the CEG inthe form of nodes, edges and properties. The nodes represent the objectsand data elements, while the edges connect two or more nodes andrepresent relationships between the connected nodes. Both nodes andedges may have properties that describe a particular instance in detail.The CEG may be displayed to the engineer at different levels of detail.The following examples will be explained with reference to anengineering design depicted in a view of an engineering tool asillustrated in FIG. 2 .

FIG. 2 is an example of a screen display of an engineering toolaccording to aspects of embodiments of the present disclosure. Thedisplay 210 depicts the interface provided to an operator as ahuman-machine interface (HMI) used to control a manufacturing system.The engineering project developed in the engineering tool also providescontrol functionality for the automation system. The interface providesdisplay elements that serve to operate control functions 207 to allowthe operator to control the system. The display 210 may be a touchdisplay allowing the operator to interact directly with the screenelements to perform functions such as starting and stopping the systemor selecting automatic or manual operating modes. In other embodiments,interaction with the control elements may occur through input devices,such as a mouse, trackball, trackpad and the like.

Navigation buttons 211 provide additional functionality to assist inoperation of the system. Objects that make up physical components of theautomation system may be displayed in a workspace region of the display210. Objects may include an industrial robot 201, a conveyor 203, alight tower 205 and an object of manufacture 209. Light tower 205provides an indication of the current operating status of the system.For example, if the green portion of the light tower is illuminated, itmay indicate that the system is currently actively operating. The lighttower alerts a user or bystander to the fact that the robot 201 or theconveyor 203 may be in motion and present a danger to health and safety.Conveyor 203 is used to transport the object of manufacture 209throughout the plant. For example, conveyor 203 may transport the objectof manufacture 209 to a workstation “manned” by industrial robot 201.Industrial robot 201 may then perform manufacturing actions on theobject of manufacture 209. When the actions of the industrial robot 201are complete, conveyor 303 may transport the object of manufacture 209to another workstation for further processing, or if manufacturing iscomplete, may transport the object of manufacture 209 for finalinspection or shipping.

FIG. 3 is a diagram of a cognitive engineering graph that isrepresentative of a graph that the cognitive engineering system maygenerate based on the design project depicted in FIG. 2 . Arrows betweenthe nodes define relationships between connected nodes. Information fromthe engineering tool, provided in this example as TIA portal 111, isused to generate the CEG 300. The engineering tool 111 includes a designproject 200 such as the project illustrated in FIG. 2 . The project 200defines an automation application for a robot with a conveyor. Theproject 200 comprises components that define functionality for a humanmachine interface (HMI) 310, which provides interaction between theautomation system with a human user or operator. The project 200 alsoprovides functionality for a programmable logic controller (PLC) 320 forcontrol of the automation system. The PLC 320 monitors operation of theautomation system and provides control of the system through varioussignals and can generate alerts that guide operation of the componentsof the system.

The HMI 310 provides a root screen 330 that is displayed to the user.Objects displayed on the screen are elements of the root screen 330. Forexample, the robot 201 display shown in FIG. 2 , is constructed fromgeometric shapes 331 including circles, ellipses, and rectangles.Buttons 333 for control of the system are also included in the rootscreen 330. The light tower 205 shown in FIG. 2 includes three coloredrectangles: a red rectangle 335 a, a yellow rectangle 335 b and a greenrectangle 335 c.

PLC 320 provides blocks for monitoring data and providing control to theautomation system. There is a first function block (FB) 321 containingthe logic required to control the robot 201 and a second FB 325 forcontrolling the conveyor. Data blocks (DB) 325, 327 store data relatingto the robot and the conveyor, respectively. The PLC 320 uses labelsthat identify the functions and properties of the components of theautomation system. The labels or tags are stored in a tag table 340. Thetags aid the PLC 320 in interacting with the display of the HMI 310through an organization block (OB) 329. Via the OB 329, the values ofthe light tower lights can be controlled and displayed to the user.Using tags 345 a for the red light, 345 b for the yellow light and 345 cfor the green light, the OB 329 sets the value at each tag. For example,the value may be a binary value indicating whether the associated lightis illuminated or dim. Using the tags 345 a, 345 b, 345 c, the color ofthe associated rectangle 335 a, 335 b, 335 c may be set to indicate eachlights status. To display the light status to the user, for example, adarker shade may be used to indicate when the light is not illuminated,while a lighter, more vibrant shade of color may be used to indicate alight that is illuminated.

The structure of the CEG is constructed to include knowledge based onhistorical data from projects that have been previously designed andanalyzed by the CEG. The past knowledge obtained by the CEG may beanalyzed to discover useful patterns. For example, through logic-basedpattern-matching or statistical machine learning, patterns may beidentified that identify certain objects or processes that may be of usein the future. When a pattern is identified, either in a supervised orunsupervised manner, it can be applied to new contexts for variouspurposes including:

-   -   validating existing designs by comparing the existing solution        with the known pattern; and/or    -   modifying or extending existing designs or creating new designs        by applying the pattern for transforming the CEG.

An example will now be illustrated where the discovery and use of alight tower such as the light tower 205 in FIG. 2 is used to explain howthe CEG may be employed for cognitive automation engineering. A lighttower 205 is a high-level concept not represented explicitly in theengineering tool 111. Nevertheless, the engineer must understand thisconcept in order to perform corresponding automation engineering tasks.To this purpose, the CEG may be used to represent the high-level conceptof the light tower and work with the concept directly. To achieve this,a first step involves the user querying the CES to determine if a lighttower is included in the engineering project 200. The CES has analyzedmultiple engineering projects from the past and stores this in arepository of past knowledge. Accordingly, the CES has learned patternsthat identify a light tower. The CES may refer to the patternattributable to a light tower in a query provided by a user that isstructured in a language such as Cypher as follows:

MATCH (n:TIA_Portal)-[rProject:Project]-> (project), (project)-[rPLC:PLC]->(plc),  (plc)-[rBlock:Block]->(block), (plc)-[rTagTable:TagTable]->(tagtable), (tagtable)-[rTag:Tag]->(tag{dir:‘Out‘}),  (project)-[rHMI:HMI]->(hmi), (hmi)-[rScreen:Screen]->(screen), (screen)-[rElement:Element]->(rectangle{type:‘Rectangle‘}), (tag)-[rChangeColor:ChangeColor]->(rectangle), (block)-[rSetValue:SetValue]->(tag) RETURN*

It should be noted that other formats or query languages may be used.The CES identifies the pattern associated with the user's request andsearches for the light tower pattern in the current CEG to find a match.In this example, a match is found. The elements of this match are shownin FIG. 4 .

FIG. 4 illustrates a pattern 400 contained in a CEG as identified in thequery above. The pattern includes the project 200 and its implementationof an HMI 310 and PLC 320. In addition, the components that representthe light tower including the labels 345 a, 345 b, 345 c from theorganization block 329 and the rectangles 335 a, 335 b, 335 c that areused in the display screen 330 of the HMI 310 are included in thepattern. Any project that contains these elements connecting in asimilar manner would be indicative that the project contains a lighttower.

FIG. 5 is an illustration of a portion of a CEG that is representativeof a light tower object. The CES creates new high-level nodes and edges(denoted by borders having broken lines) for representing the concept ofthe light tower. The new nodes include a LightTower object 501 that isassociated with a LightTowerView 503, LightTowerTags 505, andLightTowerLogic 507. The LightTowerLogic 507 is contained in the OB 329.LightTowerView 503 contains red rectangle 335 a, yellow rectangle 335 band green rectangle 335 c. LightTowerTags 505 contains the labels forred_light 345 a, yellow_light 345 b and green_light 345 c. These newstructures 501, 503, 505 and 507 are created to represent the concept ofthe light tower in an explicit manner within the CEG. This allows boththe user and the CES to conceive the same concepts and communicateeffectively about them.

The CES analyzes the CEG and infers properties about the high-levelconcept of the light tower. These properties are presented to theengineer, and he/she can request the CES to modify them. The CES candetermine the steps to carry out these modifications and can also warnthe engineer about possible problems that can arise. For the lighttower, the CES determines the number and colors of the lights in thelight tower as a property: red, yellow and green. This property may beshown to the user in a simplified form as illustrated in FIG. 6 .

FIG. 6 . is a simplified high-level view 600 of the conceptual lighttower from a CEG that can be displayed to a user. This view only showsthe most basic relevant nodes and edges in the CEG that relate to thelight tower while hiding all other information. This allows the user toconcentrate on the current task of working with the light tower.According to embodiments of this disclosure, the user may work with theconcept of the light tower by requesting the CES to add a new blue lightto the light tower. This may be accomplished simply by adding a new item“blue” to the “Lights” property of the high-level light tower element asshown in 701 of FIG. 7 .

FIG. 7 is an illustration of an updated CEG generated automatically bythe CES according to aspects of embodiments of the present disclosure.The user requests the addition of a blue light to the light tower. TheCES having previously analyzed the CEG, includes functionality toimplement the new property automatically. The CES proceeds to add a newoutput signal for the new blue light 703 adding a new graphical element705 to the view of the light tower 501 and adding control logic to theMain_OB 329 program block for setting the value of the blue lightsignal. The names of the new elements are inferred by the CES based onthe naming conventions of the existing elements: <COLOR>_light for thesignal (e.g. blue_light 703), and <COLOR>_Rectangle<NUM> for thegraphical element (e.g., BlueRectangle1 705). The CES displays thechanges to be performed in the CEG: including the updated property 701,new nodes 703, 705, and new edges for the new relationships containingthe new nodes 703, 705. The user may review the updated CEG and decideif the changes are to be applied. If so, the CES updates the CEG andexecutes all required engineering steps in the engineering system. Theupdated CEG is shown to the user (FIG. 8 ). For illustrative purposes,the newly created nodes are shown in FIG. 8 as hatched ovals.

The pattern for transforming the CEG to add the new light mayalternatively be expressed as a query for the graph database. This timewith instructions that match the context where the transformation willoccur. The instructions that create new nodes and edges in the graph maybe expressed as follows:

MATCH  (tia:TIA_Portal)-[rProject:Project]->(project), (project)-[rPLC:PLC]->(plc),  (plc)-[rBlock:Block]->(block), (plc)-[rTagTable:TagTable]->(tagtable), (tagtable)-[rTag:Tag]->(tag{dir:‘Out‘}),  (project)-[rHMI:HMI]->(hmi), (hmi)-[rScreen:Screen]->(screen), (pattern:LightTower)-[rLightTowerTags]->(tags:LightTowerTags), (pattern:LightTower)-[rLightTowerView]->(tags:LightTowerView), (pattern:LightTower)-[rLightTowerLogic]->(tags:LightTowerLogic),MERGE(newtag:blue_light{name:‘blue_light‘, dir:‘Out;, kind: ‘created’})MERGE (newrectangle:BlueRectangle1 {name:‘BlueRectangle1‘, type:‘Rectangle’, kind: ‘created’}) MERGE (tagtable)-[rTag2:Tag]->(newTag)MERGE (screen)-[rElement:Element]->(newretangle) MERGE(newtag)-[rChangeColor:ChangeColor]->(newrectangle) MERGE(block)-[rSetValue:SetValue]->(newtag) MERGE(tags)-[rContains1:Contains]->(newtag) MERGE*view)-[rContains2:Contains]->(newrectangle) RETURN *

FIG. 8 is an updated portion of a CEG after adding the new blue light tothe light tower. The updated light tower elements in the CEG are shownto the user. In this manner, the multiple effects of adding a singlelight may be easily observed:

-   -   the new output signal has been added to the default tag table        703;    -   the new graphical element for the light (BlueRectangle1) 705 has        been added to the root screen;    -   the Main_OB program block 329 now has code for setting the value        of the new output signal 703.    -   The output signal is now set to change the color of the blue        light in the HMI screen based on its signal value. (The colors        in the HMI have been chosen to represent an on/off blue light,        now shown in this view of the CEG);

Referring now to FIG. 9 , a diagram of an entire CEG after the additionof a new blue light to the light tower is shown according to aspects ofembodiments of the present disclosure. After adding the high-levelconcept structures for the light tower and the new blue light, theentire CEG is shown to the user as illustrated in FIG. 9 . Additionaldiscovery of patterns and high-level modifications may be requested bythe engineer. Also, all changes to the CEG are tracked by the CES, andthe user can request to undo or modify previous decisions. For example,the user may undo automatic changes made by the CES. The changes to beundone may reside in multiple places throughout the CES. A history ofthe changes made can be stored, and the user can request that thechanges be rolled back to a previous point in time. When changes The CESwill then analyze the CEG, identify the necessary changes and apply themaccordingly to the CEG and to the engineering tools.

In addition to the general information stored in the CEG regarding thevarious components of the system, the CES may require additionalinformation in order to make reasoning decisions about the automationproblems to be solved by the system being designed. In order to allowcomprehensive reasoning regarding the problems faced, the system needsan understanding of the physical world and not just the general terms ofthe task to be solved. A typical automation program may include inputmodules for various sensors, output modules for various actuators and ifused, one or more drive modules to control electric motors. However,this approach is missing a connection of these modules to the actualphysical devices that are the subject of the automation program. Even ifthese connections were included in some textual form in the comments ofthe program, this would not provide a reliable source of information.According to some embodiments of the present disclosure, the CEG isdeveloped to accommodate the representation of objects with varyinglevels of detail.

FIG. 11 is a block diagram of a CEG that provides varying level ofdetail for objects represented by the CEG. In a general sense, the CEGmay contain nodes that are placeholders for physical objects. Forexample, a conveyor, an industrial robot or a product of manufacture maybe represented by a node that does little more than identify the genericnature of the object. When the graph includes more information relatingto these objects, the system can be trained to provide analysis andrecommendations at the more precise level of detail. Considering theengineer and the CES have a true understanding of what a “conveyor” is,they can communicate more effectively about it. To this end, the CEG maybe configured to store information including the physical properties ofthe conveyor (e.g., from a 3D model), the kinematics of the conveyor(e.g., joins, friction), the electrical connections and the connectionto the industrial controller (e.g., PLC) along with knowledge of how atypical automation program is structured. Referring now to FIG. 11 , aconveyor 1101 is represented as a generic concept, a conveyor in generalterms is identified by a skill is provides, namely, transport 1140. Asmay be seen in FIG. 11 , a conveyor 1101 may be further defined toinclude sub-classes of a conveyor including a belt conveyor 1103 and amagnetic conveyor 1105. The subclass may be defined to contain morespecific information pertaining to a belt conveyor as opposed to amagnetic conveyor. For instance, a belt conveyor may be represented ascontaining parts 1107 that include a belt 1102, a first roller 1104, asecond roller 1106 and a motor 1109. In addition, if the layer of detaildefines the conveyor 1101 as a belt conveyor 1103, without furtherinformation, the CEG may be configured to deploy a generic automationprogram 1110 that is associated with a belt conveyor. However, the beltconveyor subclass 1103 may be represented in greater detail by defininga belt conveyor of a given type that is acquired from a given vendor1130. In this case, the CEG may be configured to include a specificautomation program 1120 for the specific conveyor of the given type andvendor 1130. Further still, the CEG can be configured to represent oneor more instances of a belt conveyor of the given type from the givenvendor 1131. By allowing for a varying degree of detail whenrepresenting objects within the CEG, the system is able to perform morein-depth analysis through machine learning techniques.

The varying detail level of the CEG can be achieved through thefollowing elements:

-   -   Abstraction and Refinement—Each class of object will be        available on several layers of abstraction. Depending on the        amount of available context information, a more generic/abstract        or a more specific/concreate variant of the object will be used.        This approach applies not only to physical objects, but for        other concepts as well, such as skills. The term skill may be        interpreted to mean just a service that some physical object can        perform for the user. For example, a conveyor can transport        things from point a to point b. Accordingly, a conveyor should        have a transport skill. In contrast, a pick-and-place robot may        also provide a transport skill but may simultaneously flip an        object over by 180 degrees.    -   Classes and Instances—If an object is used in a specific        project, an instance of the class object will be created in the        CEG. The instance may come from a class higher up in the        derivation hierarchy and only over time when more information is        available, will be the instance of a more and more specific        class and eventually represent the concrete physical object in        the shop floor.    -   Control and Simulation Behavior—Objects in the CEGH have not        only an ontological description, but at the same time an        executable description of their behavior. In the case where the        engineering system which hosts the CEG also has a simulation        environment, this behavior could also be executed in simulation        mode.

Some embodiments may be realized in the form of a graph database andaccompanying software interfaces, which implement the knowledgerepresentation functionality of the CEG while adhering to the principlesabove. While object-oriented databases and programming languages exist,they follow a basic design principle by which all objects are fullydefined at their creation meaning that they lack the flexibility ofbeing able to represent objects with increasing level of detailthroughout the object's lifecycle.

FIG. 12 is a block diagram for representing knowledge in a cognitiveengineering system. Information 1205 is received from an engineeringtool 1201. The engineering tool 1201 may receive input from a user 1203and additionally generates information in an engineering design projectbased in part on the user input 1203. The information 1205 from theengineering tool 1201 may include information relating to physicalobjects of an automation system, control objects in an automationsystem, and control programs for operating control objects and otherphysical objects in the automation system. Any information relating tothe automation system, including components contained in the system andinformation relating to the relationships between two or more componentsmay be included the information 1205 received from the engineering tool1201.

A historical record of previously designed projects may be stored in theform of a CEG. Multiple instances of previously generated CEGs 1209 maybe stored. A communications link 1211 between the stored previously CEGs1209 and the current CEG 1207 generated from on the information 1205.The information in the current CEG 1207 and the previously generatedCEGs 1209 are included in the knowledge representation 1213 of thesystem. The knowledge representation contains stored knowledge gainedfrom the experience of designers of varying experience and skill levelsthrough the design on the current project and previously designedprojects.

Machine learning 1215 may be applied to the knowledge representation todetermine design choices and practices that have been determined to besuccessful, or conversely, design choices and practices that weredetermined to be unsuccessful. Machine learning 1215 may use theknowledge representation 1213 to make recommendations to a user viaengineering tool 1201. Additionally, machine learning 1215 may examinean engineering project in the engineering tool 1201 and validate 1217the design based on prior knowledge contained in the knowledgerepresentation 1213.

FIG. 10 illustrates an exemplary computing environment 1000 within whichembodiments of the invention may be implemented. Computers and computingenvironments, such as computer system 1010 and computing environment1000, are known to those of skill in the art and thus are describedbriefly here.

As shown in FIG. 10 , the computer system 1010 may include acommunication mechanism such as a system bus 1021 or other communicationmechanism for communicating information within the computer system 1010.The computer system 1010 further includes one or more processors 1020coupled with the system bus 1021 for processing the information.

The processors 1020 may include one or more central processing units(CPUs), graphical processing units (GPUs), or any other processor knownin the art. More generally, a processor as used herein is a device forexecuting machine-readable instructions stored on a computer readablemedium, for performing tasks and may comprise any one or combination of,hardware and firmware. A processor may also comprise memory storingmachine-readable instructions executable for performing tasks. Aprocessor acts upon information by manipulating, analyzing, modifying,converting or transmitting information for use by an executableprocedure or an information device, and/or by routing the information toan output device. A processor may use or comprise the capabilities of acomputer, controller or microprocessor, for example, and be conditionedusing executable instructions to perform special purpose functions notperformed by a general-purpose computer. A processor may be coupled(electrically and/or as comprising executable components) with any otherprocessor enabling interaction and/or communication there-between. Auser interface processor or generator is a known element comprisingelectronic circuitry or software or a combination of both for generatingdisplay images or portions thereof. A user interface comprises one ormore display images enabling user interaction with a processor or otherdevice.

Continuing with reference to FIG. 10 , the computer system 1010 alsoincludes a system memory 1030 coupled to the system bus 1021 for storinginformation and instructions to be executed by processors 1020. Thesystem memory 1030 may include computer readable storage media in theform of volatile and/or nonvolatile memory, such as read only memory(ROM) 1031 and/or random-access memory (RAM) 1032. The RAM 1032 mayinclude other dynamic storage device(s) (e.g., dynamic RAM, static RAM,and synchronous DRAM). The ROM 1031 may include other static storagedevice(s) (e.g., programmable ROM, erasable PROM, and electricallyerasable PROM). In addition, the system memory 1030 may be used forstoring temporary variables or other intermediate information during theexecution of instructions by the processors 1020. A basic input/outputsystem 1033 (BIOS) containing the basic routines that help to transferinformation between elements within computer system 1010, such as duringstart-up, may be stored in the ROM 1031. RAM 1032 may contain dataand/or program modules that are immediately accessible to and/orpresently being operated on by the processors 1020. System memory 1030may additionally include, for example, operating system 1034,application programs 1035, other program modules 1036 and program data1037.

The computer system 1010 also includes a disk controller 1040 coupled tothe system bus 1021 to control one or more storage devices for storinginformation and instructions, such as a magnetic hard disk 1041 and aremovable media drive 1042 (e.g., floppy disk drive, compact disc drive,tape drive, and/or solid state drive). Storage devices may be added tothe computer system 1010 using an appropriate device interface (e.g., asmall computer system interface (SCSI), integrated device electronics(IDE), Universal Serial Bus (USB), or FireWire).

The computer system 1010 may also include a display controller 1065coupled to the system bus 1021 to control a display or monitor 1066,such as a cathode ray tube (CRT) or liquid crystal display (LCD), fordisplaying information to a computer user. The computer system includesan input interface 1060 and one or more input devices, such as akeyboard 1062 and a pointing device 1061, for interacting with acomputer user and providing information to the processors 1020. Thepointing device 1061, for example, may be a mouse, a light pen, atrackball, or a pointing stick for communicating direction informationand command selections to the processors 1020 and for controlling cursormovement on the display 1066. The display 1066 may provide a touchscreen interface which allows input to supplement or replace thecommunication of direction information and command selections by thepointing device 1061. In some embodiments, an augmented reality device1067 that is wearable by a user, may provide input/output functionalityallowing a user to interact with both a physical and virtual world. Theaugmented reality device 1067 is in communication with the displaycontroller 1065 and the user input interface 1060 allowing a user tointeract with virtual items generated in the augmented reality device1067 by the display controller 1065. The user may also provide gesturesthat are detected by the augmented reality device 1067 and transmittedto the user input interface 1060 as input signals.

The computer system 1010 may perform a portion or all of the processingsteps of embodiments of the invention in response to the processors 1020executing one or more sequences of one or more instructions contained ina memory, such as the system memory 1030. Such instructions may be readinto the system memory 1030 from another computer readable medium, suchas a magnetic hard disk 1041 or a removable media drive 1042. Themagnetic hard disk 1041 may contain one or more datastores and datafiles used by embodiments of the present invention. Datastore contentsand data files may be encrypted to improve security. The processors 1020may also be employed in a multi-processing arrangement to execute theone or more sequences of instructions contained in system memory 1030.In alternative embodiments, hard-wired circuitry may be used in place ofor in combination with software instructions. Thus, embodiments are notlimited to any specific combination of hardware circuitry and software.

As stated above, the computer system 1010 may include at least onecomputer readable medium or memory for holding instructions programmedaccording to embodiments of the invention and for containing datastructures, tables, records, or other data described herein. The term“computer readable medium” as used herein refers to any medium thatparticipates in providing instructions to the processors 1020 forexecution. A computer readable medium may take many forms including, butnot limited to, non-transitory, non-volatile media, volatile media, andtransmission media. Non-limiting examples of non-volatile media includeoptical disks, solid state drives, magnetic disks, and magneto-opticaldisks, such as magnetic hard disk 1041 or removable media drive 1042.Non-limiting examples of volatile media include dynamic memory, such assystem memory 1030. Non-limiting examples of transmission media includecoaxial cables, copper wire, and fiber optics, including the wires thatmake up the system bus 1021. Transmission media may also take the formof acoustic or light waves, such as those generated during radio waveand infrared data communications.

The computing environment 1000 may further include the computer system1010 operating in a networked environment using logical connections toone or more remote computers, such as remote computing device 1080.Remote computing device 1080 may be a personal computer (laptop ordesktop), a mobile device, a server, a router, a network PC, a peerdevice or other common network node, and typically includes many or allof the elements described above relative to computer system 1010. Whenused in a networking environment, computer system 1010 may include modem1072 for establishing communications over a network 1071, such as theInternet. Modem 1072 may be connected to system bus 1021 via usernetwork interface 1070, or via another appropriate mechanism.

Network 1071 may be any network or system generally known in the art,including the Internet, an intranet, a local area network (LAN), a widearea network (WAN), a metropolitan area network (MAN), a directconnection or series of connections, a cellular telephone network, orany other network or medium capable of facilitating communicationbetween computer system 1010 and other computers (e.g., remote computingdevice 1080). The network 1071 may be wired, wireless or a combinationthereof. Wired connections may be implemented using Ethernet, UniversalSerial Bus (USB), RJ-6, or any other wired connection generally known inthe art. Wireless connections may be implemented using Wi-Fi, WiMAX, andBluetooth, infrared, cellular networks, satellite or any other wirelessconnection methodology generally known in the art. Additionally, severalnetworks may work alone or in communication with each other tofacilitate communication in the network 1071.

An executable application, as used herein, comprises code ormachine-readable instructions for conditioning the processor toimplement predetermined functions, such as those of an operating system,a context data acquisition system or other information processingsystem, for example, in response to user command or input. An executableprocedure is a segment of code or machine-readable instruction,sub-routine, or other distinct section of code or portion of anexecutable application for performing one or more particular processes.These processes may include receiving input data and/or parameters,performing operations on received input data and/or performing functionsin response to received input parameters, and providing resulting outputdata and/or parameters.

A graphical user interface (GUI), as used herein, comprises one or moredisplay images, generated by a display processor and enabling userinteraction with a processor or other device and associated dataacquisition and processing functions. The GUI also includes anexecutable procedure or executable application. The executable procedureor executable application conditions the display processor to generatesignals representing the GUI display images. These signals are suppliedto a display device which displays the image for viewing by the user.The processor, under control of an executable procedure or executableapplication, manipulates the GUI display images in response to signalsreceived from the input devices. In this way, the user may interact withthe display image using the input devices, enabling user interactionwith the processor or other device.

The functions and process steps herein may be performed automatically orwholly or partially in response to user command. An activity (includinga step) performed automatically is performed in response to one or moreexecutable instructions or device operation without user directinitiation of the activity.

The system and processes of the figures are not exclusive. Othersystems, processes and menus may be derived in accordance with theprinciples of the invention to accomplish the same objectives. Althoughthis invention has been described with reference to particularembodiments, it is to be understood that the embodiments and variationsshown and described herein are for illustration purposes only.Modifications to the current design may be implemented by those skilledin the art, without departing from the scope of the invention. Asdescribed herein, the various systems, subsystems, agents, managers andprocesses can be implemented using hardware components, softwarecomponents, and/or combinations thereof.

What is claimed is:
 1. A method for representing knowledge in acognitive engineering system (CES) comprising: receiving informationrelating to an automation engineering project from an engineering tool;storing the received information in a cognitive engineering graph (CEG)comprising a plurality of nodes representative of an element of theautomation engineering project and at least on edge connecting two ofthe nodes, the at least one edge representative of a relationshipbetween the connected nodes; and storing a plurality of previouslygenerated CEGs representative of other prior automation engineeringprojects; and establishing a communication path between the CEG storingthe received information and the plurality of previously generated CEGs.2. The method of claim 1, further comprising: applying machine learningto the stored CEG based on the received information and the storedplurality of previously generated CEGs.
 3. The method of claim 1,further comprising: analyzing the CEG based on the received informationto identify at least one pattern that is representative of a givenobject of interest from the automation engineering project.
 4. Themethod of claim 1, further comprising: automatically by the CES, addingan element to the CEG based on the received information and on a queryfrom a user.
 5. The method of claim 4, further comprising: performing anundo action by the CES at a request of a user that removes the elementthat was automatically added to the CEG.
 6. The method of claim 1,wherein the CEG from the received information includes nodes thatrepresent physical objects in the automation engineering project.
 7. Themethod of claim 6, wherein the CEG from the received informationincludes nodes that represent an automation program for controlling acorresponding physical object in the automation engineering project. 8.The method of claim 1, wherein the CEG from the received informationincludes at least one node that represents a human machine interface(HMI).
 9. The method of claim 1, wherein the CEG from the receivedinformation includes at least one node that represents a programmablelogic controller (PLC).
 10. The method of claim 1, further comprising:comparing the CEG based on the received information and the storedplurality of previously generated CEGs; and validating a design for theautomation engineering project based on the comparison.
 11. The methodof claim 1, further comprising: comparing the CEG based on the receivedinformation and the stored plurality of previously generated CEGs; anddetermining a proposed course of action for the user to perform in theautomation engineering project based on the comparison; andcommunicating the propose course of action to the user.
 12. A system forproviding a knowledge representation in a cognitive engineering system(CES) comprising: a computer-based engineering tool for providing atleast one of designing, programming simulation and testing of anautomation system; a cognitive system in communication with thecomputer-based engineering tool comprising: a knowledge extractionmodule for identifying and storing information contained in a project ofthe computer-based engineering tool and from data received from aphysical automation system; a machine learning module for analyzingknowledge extracted by the knowledge extraction module and identifyingcharacteristics of the automation system; an inductive programmingmodule of automatically generating control programs for the automationsystem based on the stored information from the knowledge extractionmodule; and a knowledge representation comprising a cognitiveengineering graph (CEG), the CEG comprising a plurality of nodesrepresentative of an element of the automation engineering project andat least on edge connecting two of the nodes, the at least one edgerepresentative of a relationship between the connected nodes.
 13. Thesystem of claim 12 further comprising: a computer memory storing aplurality of CEGs from previously designed projects in communicationwith the machine learning module for analyzing past knowledge.
 14. Thesystem of claim 12 further comprising: a feedback module for providinginformation from the cognitive system to a user.
 15. The system of claim14, wherein the feedback module is configured to provide the user with adesign recommendation for the automation engineering project based on anoutput from the machine learning module.
 16. The system of claim 12,further comprising: a communication channel between a physicalautomation system and the knowledge extraction module for extractingoperations data from the automation system for analysis by the cognitivesystem.
 17. The system of claim 12, further comprising an automatedreasoning module in communication with the knowledge representation andthe machine learning module, the automated reasoning module configuredto automatically add a component to the automation engineering projectbased on the knowledge representation and the machine learning module.18. The system of claim 12, the CEG comprising: at least one noderepresentative of a physical element of an automation system.
 19. Thesystem of claim 12, the CEG comprising: at least one node representativeof a human machine interface (HMI) for an automation system.
 20. Thesystem of claim 12, the CEG comprising: at least one node representativeof a programmable logic controller (PLC) for an automation system.